Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach

Stochastic Dynamic Programming with Markov Chains for Optimal Sustainable Control of the Forest Sector with Continuous Cover Forestry

We present a stochastic dynamic programming approach with Markov chains for optimal control of the forest sector. The forest is managed via continuous cover forestry and the complete system is sustainable. Forest industry production, logistic solutions and harvest levels are optimized based on the sequentially revealed states of the markets. Adaptive full system optimization is necessary for co...

Singularly Perturbed Markov Chains : Limit Results and Applications

This work focuses on time-inhomogeneous Markov chains with two time scales. Our motivations stem from applications in reliability and dependability, queueing networks, financial engineering and manufacturing systems, where two-time-scale scenarios naturally arise. One of the important questions is: As the rate of fluctuation of the Markov chain goes to infinity, if the limit distributions of su...

A new Approach to the Multi-Objective Control of Linear Singular Perturbation Systems

A novel technique for a class of singular boundary value problems

In this paper, Lagrange interpolation in Chebyshev-Gauss-Lobatto nodes is used to develop a procedure for finding discrete and continuous approximate solutions of a singular boundary value problem. At first, a continuous time optimization problem related to the original singular boundary value problem is proposed. Then, using the Chebyshev- Gauss-Lobatto nodes, we convert the continuous time op...

Cascade Markov Decision Processes: Theory and Applications

This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine the optimal performance and feedback controls, while some other cases are shown to lead to singular optimal control problems which are more difficult to solve....